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Introduction 

Diverse graduate studies penneate the university landscape. Master’s work is often 
considered the essential, terminal degree for a variety of professional programs that derive their 
value from accreditation. Respective accrediting organizations, associations, or agencies issue 
standards for programs to follow and competencies for students to achieve. Accreditation is 
awarded or continued according to how well the master’s programs demonstrate compliance to 
standards and how well students demonstrate achievement of competencies expressed as learning 
outcomes. Even though many indicators of graduate program quality are already in place, 
effectively demonstrating student achievement of learning outcomes remains a challenge. This 
study demonstrates how LIS educators accepted the challenge by creating a survey instrument 
aligned to program standards, competencies, and courses to assess student learning and isolate 
areas for program and curriculum improvement required for accreditation. 

Literature Review 

Despite being associated with accredited universities, professional graduate programs 
typically fall under the additional scrutiny of accrediting bodies charged with oversight of 
specific professions. For example, the Council on Social Work Education (CSWE) monitors 
social work education, specifically the MSW, and the American Library Association (ALA) 
oversees various master’s programs for library science throughout the United States and Canada 
(Applegate, 2006; Calderon, 2013). Likewise, the National Association of Schools of Public 
Affairs & Administration (NASPAA) reviews graduate public affairs programs for adherence to 
specific standards (Powell, Saint-Germain, & Sundstrom, 2014). Evidence offered by those in 


1 



charge of these programs often included items that point to the overall satisfaction or quality of 
the program perceived by others, generally students, alumni, and employers. Some even pointed 
to the U.S. News and World Report rankings for the program. This survey and ranking data, 
though, were seen as questionable or insufficient. For instance, while current student entrance 
and exit survey data can be regarded as reflective of student experiences, alumni data can be 
outdated or incomplete. Also, the methodology behind college, university, or program rankings 
is under scrutiny (Applegate, 2006; Rogers & Goktas, 2010). Consequently, even though survey 
data and faculty accomplishments might be referenced by programs seeking accreditation or 
reaccreditation, the program now must provide evidence of a continuous outcomes assessment 
cycle resulting in an improved curriculum to enhance learning. Assessment of student learning 
and mastery of specific standards and competencies are primary (Applegate, 2006; Calderon, 
2013; Lattuca, Terenzini, & Volkwein, 2006; Pinto, Fernandez-Ramos, Sanchez, & Meneses, 
2013; Powell et al., 2014; Rogers & Goktas, 2010). 

A clear, evidenced-based approach to learning outcomes assessment for professional 
graduate programs has not been previously well defined in the literature (Applegate, 2006; 
Brown & Benson, 2005; Burke & Snead, 2014; Rogers & Goktas, 2010) and a recent search of 
the literature resulted in the same findings. Evident in the literature, though, was a noticeable 
tension between the types of instruments used for a true, effective measurement of student 
learning. Direct measures, such as tests or practical experiences, are accepted by higher 
education as true, valid demonstrations of what content actually has been learned and indirect 
measures are thought of as only indicators that some learning has occurred, even though they 
provide students the opportunity to think seriously about their learning (Applegate, 2006; 
Calderon, 2013; Suskie, 2004). One form of indirect measures frequently used by professional 
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graduate programs is the self-assessment survey (Applegate, 2006; Calderon, 2013; Pinto et al., 
2013). Students are asked to evaluate their knowledge and abilities related to the program 
specific competencies issued by the program’s accrediting body, thereby calling for a high level 
of self-awareness and integrity on the part of the student. 

Gathering information from the student’s perspective can be valuable since it can indicate 
levels of motivation and confidence (Pinto et al., 2013). Indirect measures, especially the self- 
assessment survey, however, are frequently viewed as problematic. While self-assessment 
instruments can work effectively when students collect and reflect on their performance data on 
an ongoing basis and compare it to data received from direct measures (Colthart et al., 2008), 
studies revealed that students, especially weaker students, tend to overestimate their abilities 
(Calderon, 2013; Colthart et al., 2008; Pinto et al., 2013). Conversely, stronger students tend to 
underestimate their abilities (Colthart et al., 2008). One reason offered for these tendencies is the 
degree of the students’ metacognitive abilities. Researchers concluded that weaker students are 
incapable of recognizing their true abilities (Colthart et al., 2008). Calderon’s (2013) findings 
offered another possibility by suggesting that direct and indirect instruments really measure two 
different constructs. Direct measures demonstrate actual learning while indirect self-assessment 
surveys indicate a student’s learning experiences. The perception of learning is viewed 
separately from actual attainment of skills and competencies and the survey may really reflect 
different aspects, such as satisfaction with more social or emotional experiences during the 
program. Calderon (2013) concluded that further research is needed to identify the factors related 
to a student’s learning experiences. 

Some of the authors reviewed in the literature were investigating the use of self- 
assessment as one way to meet accreditation requirements (Calderon, 2013; Pinto et al., 2013; 
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Rogers & Goktas, 2010). Others embraced the need for a variety of assessments, both direct and 
indirect, depending on whether formative or summative data was desired. Student artifacts 
produced during different stages of the program also proved beneficial (Applegate, 2006; 
Calderon, 2013; Suskie, 2004). Years ago, Brown and Benson (2005) found value in capstone 
experiences, whereby students engaged in active learning and applied theory to practice in the 
chosen profession. In many respects, today’s internships are rooted in this practice and remain a 
valuable way for students to demonstrate their learning. 

One conclusion that may be obvious for accredited professional graduate programs is that 
a variety of assessment measures should provide a complete picture of student learning. The next 
steps in the cycle, data reviewed at established intervals and adjustments to the curriculum, can 
easily follow. Improvement in student learning should be the logical result, creating an effective 
assessment plan to satisfy the accrediting body. However, this is not always the case or easily 
accomplished by many professional graduate programs. Since entrance requirements for 
graduate school are demanding and call for a GPA of 3.0 or better, the student population is 
usually composed of high performing students with strong scholastic aptitude. For example, no 
discipline specific requirements exist and students from all backgrounds can pursue a library 
science degree. The main requirement is the GPA of 3.0 or better. This leads to the question: 
How can a program demonstrate a continuous cycle of assessment leading to improved student 
learning with a student population that is composed of above average, high performers? 
Background 

An ALA accredited graduate library science program in eastern United States maintains a 
variety of measures for overall program evaluation for accreditation. Data from entrance and exit 
surveys of current students, alumni surveys, and employer surveys are gathered and reviewed 
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throughout the seven year reaccreditation cycle. Assessment of student learning is ongoing and a 
number of different measures have been attempted, such as portfolios, core course exam, 
standardized tests, grades, and authentic assessments. The result was always the same: the 
students performed well, frustrating efforts to show improvement in student learning. 
Consequently, another attempt was made at creating a more meaningful assessment plan during 
the current accreditation period. 

Enhancing the Plan and Developing the Multi-Subject Learning Survey (MSLS) 

The development of the assessment plan for the curriculum and learning outcomes, which 
includes the MSLS, matured over a decade (Department of Library Science, 2010). Previous 
assessment schemes included the use of portfolios, standardized tests, grades, and authentic 
assessments. Each of these techniques demonstrated both promise and weaknesses. The use of 
portfolios had the benefit of authentic assessment but was not implemented because the parent 
university declined to increase the overall number of credits for the program which was needed 
for portfolio review. Administration of a standardized test demonstrated promise but was not 
able to be implemented at the time due to issues with the outside testing agency. Grades for 
course specific authentic assessment artifacts proved not to provide useable data given the 
limited grade range in graduate level courses. A more comprehensive approach to an assessment 
plan was developed to incorporate a range of evaluative techniques. 

The current assessment plan includes: 

• pre- and post-test administration of the MSLS; 

• review of competencies for the program with new students during orientation; 

• incorporating competency review in the mandatory advising process each semester; 

• a capstone experience including artifact collection; 
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• an employer survey based on the competencies; 

• incorporation of assessment findings into the annual program planning process. 

In addition, considering the use of a standardized test instrument and integration of the 
assessment process into an analytics software program are under review. This plan incorporates a 
number of the earlier elements in addition to newly developed assessment techniques. 

One of the new elements to be developed was a survey instrument based on learning 
competencies. The development of the survey included a review of the learning competencies for 
the program, mapping accreditation standards to the curriculum/learning objectives and 
designing and testing the MSLS instrument. Since the program has a specialty accreditation body 
the program standards required by the accrediting body were reviewed for application to learning 
competencies. An extensive project was completed to map the competencies to the current 
curriculum and learning objectives to be sure that all competencies were covered within the 
program and to note specifically which courses were required to cover which competencies. 

Once these elements were completed, the survey instrument was developed to assess student 
mastery of the competencies during the program. 

This portion of the planning process included development of the MSLS instrument, 
piloting of the instrument, and deployment procedures. The overall goal of the MSLS was to 
determine perceived knowledge level specific to program competencies by students who have 
completed the program. The survey questions were developed by using accreditation standards, 
competencies developed by professional associations and a review of the question by subject 
experts. The structure of the survey instrument developed was inspired by the Technological 
Pedagogical Content Knowledge (TPACK) instrument (Koehler & Mishra, 2008). TPACK is a 
self-assessment instrument designed to measure teacher preparation (Archambault & Crippen, 
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2009). The TPACK survey structure which queries the respondent’s self-assessment of ability in 
domain specific competencies was modeled. 

Several elements were considered to establish the construct validity of the instrument, 
that is, the extent to which inferences from the test’s scores accurately reflect the construct that 
the test is claimed to measure (Isaac & Michael, 1995). To develop the survey questions, the 
initial touchstone was the specialty accrediting body’s standards. Those standards were then 
compared to the Competency Index for the Library Field (Gutsche & WebJunction, 2009). This 
publication is a compilation of competency lists from a variety of professional associations, 
library practitioners and leaders including the American Library Association, the Special Library 
Association and subject specialists (Gutsche & WebJunction, 2009, pp. iii - iv). This approach 
had the advantage of incorporating significant expert opinion in the competency development. 
The competencies from the Index that correlated to ALA program standards were selected for 
inclusion in the MSLS. There were significantly more competencies included in the Index than 
were matched to accreditation standards and so selected for inclusion in the survey instrument. 
The selected competencies were then reviewed by six knowledgeable subject experts to ensure 
that items used were relevant, complete and structured appropriately. The use of competencies 
developed by professional organizations (including the accreditation body), practitioners and 
subject experts were used to establish an acceptable level of content validity. 

The instrument was then piloted to demonstrate that the respondents are correctly 
interpreting what each item is asking. Piloting of the instrument was undertaken by using a think- 
aloud with library science faculty, administration to a test group of library science students and 
administration to a control group of student who were not in a library science program. The think 
aloud pilot involving library science faculty involved interviewing participants while they read 


7 



and answer the questions. The faculty explained what they were thinking as they went through 
each of the questions on the instrument. Reponses were compared to ensure that the questions 
were being interpreted in the same way. The instrument was then administered to 45 students 
enrolled in a graduate level library science research methods class. The survey was administered 
to the students and then the students evaluated the instrument. The last part of the pilot process 
involved administering the survey to a graduate level group of students who were not enrolled in 
a library science program. This step established that scores from students not in a library science 
program deviated extensively from students in a library science program which was the expected 
outcome on a content dependent instrument. Changes to the instrument were made based on the 
pilot testing. 

In addition, the parent institution holds regional accreditation. As part of the preparation for 
an institution-wide accreditation review, the university brought in an assessment consultant. The 
consultant considered the following elements in reviewing the plan: 

• means of assessment are feasible and appropriate given the time and resources available; 

• multiple means of assessment are used; 

• sufficient data is collected; 

• assessment results are matched to intended outcomes; 

• data collected are meaningful, valuable and accurate. 

The department’s assessment plan was reviewed in detail by the consultant who considered the 
plan to be well structured and effective (D.G. Underwood, personal communication, February 
24, 2014). The consultant review concluded the development, testing and review of the 
instrument. Deployment of the survey was next considered. The administration of the instrument 
was written into course descriptions within the program. The instrument is administered as a 
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diagnostic assessment of students entering the program and, therefore, is included in the required 
introductory course. The MSLS is administered again as a post-test assessment at the end of the 
program in the mandatory capstone course. 

During the time of development, the parent university became involved in the Association of 
American Colleges and Universities Liberal Education and America’s Promise (LEAP) 
initiative. Elements of that process were incorporated into the assessment plan. LEAP concepts 
used for campus assessments include: 

• orientation for students; 

• diagnostic assessment; 

• advisor/plan of study; 

• alignment of curriculum/advisor review; 

• capstone experience; 

• post-test survey. 

These elements were included in the assessment plan. The competencies for the program used in 
the MSLS are reviewed during the new student orientation process. The pre-test administration 
of the MSLS occurs at the beginning of the introductory course to the program and provides a 
diagnostic assessment. Progress on attainment of the competencies is reviewed with each student 
as part of the mandatory advising session held each semester prior to registration. Since a 
capstone course is required as part of the program the MSLS is administered as a post-test during 
the capstone course. In addition, a research paper is required in the capstone course and the paper 
is collected and evaluated as an evaluative artifact. The results of the assessment plan are used 
during the annual program review to inform curricular changes. 


9 



The overall assessment plan for the program includes administration of the MSLS ; 
incorporation of the learning competencies in orientation and advising; a capstone experience 
which includes a major research paper used as an assessment artifact; an employer survey which 
queries graduates’ competency; and a process for incorporating the collected data into an annual 
program review. The MSLS assessment process addresses and evaluates the educational learning 
outcomes portion of the program. 

Data Analysis 

After piloting the MSLS instrument during summer 2012, the library science department 
decided to administer it to students enrolled in LS504, Introduction to the Information 
Professions, and one of the capstone courses, LS570, the internship course, or LS600, the 
research project course, using a pre-test/post-test design. Responses to the 35 items could range 
from 1 to 6, 1 indicating expert ability and 6 indicating no ability as shown in Table 1. 

Table 1 

Item Responses: 


Student response to item Value for computing mean 


Expert 1 

Very strong 2 

Competent 3 

Weak 4 

Very weak 5 

No ability 6 
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The pre-test results from Fall 2012 through Summer 2013 were downloaded from each 
Desire2Leam (D2L) course site and uploaded to the D2L accreditation site maintained by the 
department. The responses to the 35 survey items for each of the 124 participants were entered 
into SPSS for analysis. Descriptive statistics and item response frequencies were computed. 
Category One: Foundational Principles of Librarianship 

This category was designed to capture students’ self-assessment of their awareness of and 
ability with basic principles of librarianship. Items included familiarity with the ALA Bill of 
Rights and Code of Ethics, intellectual freedom, freedom of information, history of libraries and 
their role in society, the different types of libraries, the laws related to libraries, communicating 
the value of libraries to key stakeholders, contributing to problem solving, and the ability to 
communicate verbally and in writing. Table 2 records the mean scores for the nine items in this 
category. 

Table 2 

Mean Scores for Category One: Foundational Principles 


Items: Knowledge of/ Ability in 


Mean Scores 


1 . Relevance of A L A Bill of Rights/Code of Ethics 4. 1048 

2. Intellectual freedom/freedom of information 3.0242 

3. History of libraries/role in society 3.2016 

4. Types of libraries 2.6452 

5. Openness to new ideas/current trends 2.4435 
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6. Know/ Apply laws pertaining to libraries 


3.4194 


7. Communicate value of libraries to community 3.0161 

8. Contribute to problem-solving 2.3226 

9. Communicate verbally/in writing 2.0806 


The responses for items 1, 2, 6, 8, and 9 were interesting. Many students (61.3%) indicated that 
they had little or no familiarity with or relevance of the ALA Bill of Rights and Code of Ethics 
while even more (75%) reported their ability to promote intellectual freedom and freedom of 
information. Almost 60 % stated that they felt competent or very strong about their abilities to 
understand and apply laws pertaining to libraries and 96.8 percent of those surveyed believed 
strongly in their ability to contribute to problem-solving and finding mutually acceptable 
solutions. Almost all of the students (99.2%) reported that they could communicate openly and 
directly, either verbally or in writing. 

Category Two: Information Resources 

This category elicited responses relating to abilities in collection development and 
management. The three items addressed the ability to define policies, procedures, and criteria 
concerning the selection, evaluation, deselection, replacement, conservation, and preservation of 
library materials. Table 3 reflects the mean scores for this category. 

Table 3 

Mean Scores for Category Two: Information Resources 


Items: Knowledge of/ Ability in 


Mean Scores 
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10. Recommend policies/procedures for selection 


3.4758 


11. Collection development/acquisition process 3.6129 

12. Define criteria for collection development cycle 3.7339 


Students responded similarly to all three items. Approximately 50 percent believed they were 
competent or better with regard to the principles of collection development and management and 
approximately 50 percent felt less confident about their abilities to define criteria for the entire 
collection development cycle. 

Category Three: Organization of Information 

Category Three focused on the general structure and nature of the principles of 
information organization, systems, software, and bibliographic control standards including the 
ability to recognize and implement emerging trends. This section was comprised of three items 
and Table 4 summarizes the mean scores. 

Table 4 

Mean Scores for Category Three: Organization of Information 


Items: Knowledge of/ Ability in 


Mean Scores 


13. Application of bibliographic control standards 4.2903 

14. Current trends in bibliographic control 3.7661 

1 5 . Structure of information organization principles 3 .49 1 9 


13 



Not surprisingly, students (76.6%) acknowledged their weakness with regard to application of 
national and international bibliographic control standards to organize library materials. Almost 
half (48.4%) felt they could stay abreast of developing trends affecting bibliographic control and 
resource management. They (57.3%) even felt more confident in their ability to understand the 
structure and importance of information organization principles and systems. 

Category Four: Information Technology 

Category Four addressed information technology for libraries. Students were asked to 
identify their abilities to assess technology trends that affect the library and its users and to 
advise stakeholders concerning appropriate courses of action for technology. They were also 
directed to think about their ability to articulate and apply library policies addressing privacy, 
intellectual freedom, and ethical uses of technology. Finally, students were encouraged to reflect 
on their ability to analyze community needs for current and future trends in technology. The 
mean scores for the three items in this category are shown in Table 5. 

Table 5 

Mean Scores for Category Four: Information Technology 


Items: Knowledge of/ Ability in 


Mean Scores 


16. Assess technology trends relevant to libraries 3.5565 

17. Articulate and apply technology policies 3.4597 

18. Analyze community needs for technology 3.3468 
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Over half of the students surveyed (55.6%) responded positively to the ability to assess 
technology trends for libraries, to advise on appropriate courses of action, and to articulate and 
apply policies concerning privacy, intellectual freedom, and the ethical use of technology. 
Slightly more (61.3%) feel competent or better when it comes to analyzing community needs for 
current and future technology trends. 

Category Five: Reference 

Category Five dealt with reference services. The seven items in this section addressed the 
student’s ability to perform an effective reference interview, to understand the information- 
seeking behavior and needs of users from diverse communities, to demonstrate strong 
interpersonal communication skills, to recognize opportunities for library instruction, to identify 
and analyze target audiences for program development, to respond appropriately to diversity and 
cultural differences of users, and to perform outcome -based evaluations to measure the 
effectiveness of library programs. Table 6 itemizes the mean scores. 

Table 6 

Mean Scores for Category Five: Reference 


Items: Knowledge of/ Ability in 


Mean Scores 


1 9 . Effective reference interviewing skills 3.1613 

20. Address information-seeking behavior/needs 3.2823 

of diverse patrons 

21. Demonstrate interpersonal communication skills 1.9597 

22. Identify opportunities for instruction 2.5645 
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23. Identify, analyze target audiences 


3.4274 


24. Respond appropriately to diverse users 2.4758 

25. Perform outcome -based evaluations 3.6532 


Item responses for 21, 22, and 24 showed little room for improvement according to the students. 
Almost all (99.2%) expressed strong interpersonal communication skills, including active 
listening. Most (87.9%) felt they were quite capable in identifying opportunities for instruction to 
empower users in their own information-seeking abilities and in responding appropriately to 
diversity and cultural differences among patrons (88.7%). Fewer students (71%) claimed 
competencies in reference interviewing skills and in addressing the information-seeking 
behaviors and needs from diverse communities (59.7%), drawing some attention to the higher 
percentage reported for appropriately responding to cultural differences among patrons. Students 
reported the least ability in identifying and analyzing target audiences for programming (54%) 
and in performing outcome -based evaluations to measure the effect of various programs on users 
(48.4%). 

Category Six: Research 

The two items in Category Six highlighted basic research principles. Students were 
surveyed on their abilities to acknowledge the fundamentals of quantitative and qualitative 
research methods and to recognize, interpret, and use research literature. Table 7 lists the mean 
scores for each item. 

Table 7 

Mean Scores for Category Six: Research 
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Items: Knowledge of/ Ability in 


Mean Scores 


26. Identify/articulate fundamentals of quantitative 3.7500 

and qualitative research methods 

27. Identify, analyze, and use research literature 2.8387 


Less than half of the students (45.2%) expressed the ability to identify and articulate the 
fundamentals of quantitative and qualitative research methodology and designs, which is 
basically in agreement with their reported ability in conducting outcome-based evaluations in 
Category Five. However, noticeably more students (82.3%) claimed competency or better for 
recognizing, analyzing, and using research literature. This particular finding is definitely 
puzzling given their declared inability to recognize basic research fundamentals. 

Category Seven: Lifelong Learning 

This category emphasized a commitment to continuing education, stressing the 
importance of information literacy in libraries. Students responded to three items emphasizing 
the importance of lifelong professional growth and the need for ongoing information literacy 
initiatives in libraries. Table 8 shows the mean scores for this section. 

Table 8 

Mean Scores for Category Seven: Lifelong Learning 


Items: Knowledge of/ Ability in 


Mean Scores 


28. Importance of lifelong leaming/continuing education 2.0323 
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29. Scope/importance of information literacy/define goals 3.1371 

30. Understand/apply instructional design principles 3.7016 


Not surprisingly, most students (97.6%) understood the importance of lifelong learning in library 
work and pursuing personal and professional growth through continuing education. A little more 
than half (62.9%) reported an understanding of the importance of information literacy and the 
ability to define information literacy goals for a library. Slightly less than half (47.6%) 
acknowledged the ability to apply basic instructional design principles to design training. 
Category Eight: Administration 

The last category was aimed at determining students’ ability for administration and 
management of libraries. Five items were included in this section and were designed to elicit 
responses relating to developing and evaluating budgets, to understanding and ensuring 
compliance of human resource policies and procedures, to developing and applying methods to 
measure quality of library services, to identifying opportunities to cooperate with other libraries 
or community organizations, and to fostering an environment based on integrity and high ethical 
standards. The mean scores are itemized in Table 9. 

Table 9 

Mean Scores for Category Eight: Administration 


Items: Knowledge of/ Ability in 


Mean Scores 


3 1 . Develop/evaluate library budgets 4.3468 

32. Understand/ensure compliance to human 3.1048 
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resource policies and procedures 


33. Develop/apply methods to evaluate services 3.7097 

34. Identify opportunities to cooperate with other 3.4677 

libraries/community organizations 

35. Ability to foster environment based on 2.3790 

integrity and high ethical standards 


Students (71%) reported a definite weakness concerning the development and evaluation of 
library budgets. Students (70.2%) felt better about their abilities to understand and comply with 
human resource policies and procedures. Less than half (46%) expressed competence in 
developing and applying methods to measure the quality and value of library services. More than 
half (58.1%) believed they could cooperate with other libraries or community organizations to 
share information resources and most (89.5%) believed in their ability to foster an environment 
based on integrity and high ethical standards. 

Discussion 

Pre-test results for Category One on Foundational Principles appear to indicate 
unfamiliarity with key ALA documents for libraries. The differences in the response frequency 
for the first two items suggest that students believe they can promote intellectual freedom and 
freedom of information but do not necessarily associate it with the Library Bill of Rights and 
Code of Ethics. The topics should definitely remain in the curriculum, particularly in the 
introductory course for the program. It is unclear as to how the students would have gained 
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knowledge about laws pertaining to libraries unless, perhaps, they have experience working in a 
library. Even so, the knowledge would be somewhat limited to a specific task or practice. Their 
confidence in problem-solving and communicating effectively is admirable but remains to be 
demonstrated through coursework and real-life situations. 

Student responses for Category Two concerning collection development of information 
resources could relate to the paraprofessional experiences of the students. Many MSLS students 
work in various positions in libraries when they are enrolled in the program. This could 
contribute to their perception that they know something about collection development, especially 
with regard to carrying out established selection policies and the acquisition process. 

Generally, the results for Category Three on the Organization of Information are not 
surprising. Knowledge and ability in bibliographic control standards would not be expected for 
new students beginning the program, even though some could have some paraprofessional 
cataloging experience. Also, understanding the importance of the principles of information 
organization and of staying abreast of new developments are traits expected of new students 
entering the field of library and information science. 

The expressed knowledge and ability in Category Four on Information Technology 
suggests that students are capable of assessing technology trends for libraries, developing 
policies, and analyzing community needs. The authors know from student advising experience, 
however, that students procrastinate in taking the required technology course, frequently 
indicating a dread or fear of technology. This initial positive perception might flow from 
successful experiences as a library user and the belief that they know library technologies. 

Apparent inconsistencies are noted in Category Five on Reference. Students indicate 
competence or very strong ability in interpersonal communication skills, identifying 
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opportunities for instruction, and responding appropriately to diverse users. Lower abilities are 
expressed, though, for reference interviewing skills, addressing information needs of diverse 
users, and in identifying target audiences for programming. Students perceive even less ability in 
performing outcome -based evaluations for library programs. Students believe they possess the 
basic and beginning skills for reference and instruction and acknowledge the need to develop 
them further through coursework and applicable experiences. 

Additional inconsistencies are seen in Category Six: Research. Student inability to 
articulate the fundamentals of quantitative and qualitative research is understandable and also 
matches the weakness identified in Category Five concerning performing outcome -based 
evaluations of programs. The fact that students indicated competency or better with regard to 
analyzing and using research is somewhat troubling since it is difficult to comprehend how they 
can analyze and use it if they cannot express the fundamentals of research methodologies. This 
point reinforces the need to emphasize basic research design when reviewing relevant article 
literature in coursework. 

Not surprisingly, students acknowledge the importance in lifelong learning as expressed 
in Category Seven. Students are competent in discerning the importance of information literacy 
but express less competence when it comes to applying instructional design principles for 
training. This reinforces the decision to add an instructional strategies course to address how 
students approach information literacy instruction. 

Finally, key results are noted in Category Eight on Administration. Understandably, 
students report a definite weakness in developing and evaluating library budgets. Weakness was 
also felt with regard to developing and applying methods to measure the value of library services 
and programs. This relates to what was indicated in Categories Five and Six concerning the 


21 



inability to formulate outcome -based program evaluation and to express fundamentals of basic 
research design. The fact that students strongly perceive their ability to foster an environment 
based on integrity and high ethical standards could explain why they report competence in 
complying with human resource policies and procedures despite the lack of administrative 
opportunities. 

Conclusion 

Data analysis of post-test results is warranted to determine any significant changes in 
student responses. Data are being collected as students pursue their capstone course, the timing 
of which varies according to their individual schedules. Curriculum indications from the pre-test 
results support maintaining the content of the introductory course to include topics concerning 
the Library Bill of Rights, the ALA Code of Ethics, and other relevant material; reinforcing 
research methodologies and interpreting research literature at every opportunity; continuing with 
the instructional strategies elective; and maintaining the management course content with regard 
to budget and human resource policies and procedures. Additional indications support the need 
to maintain instruction on the interpretation of bibliographic standards for organizing and 
cataloging information; introducing developing areas for bibliographic standards, such as the 
Resource Description and Access (RDA) framework; continuing with reference and instruction 
topics, such as interviewing and programming skills; and expanding student knowledge of 
emerging library technologies through the required technology course. 

Use of the iSkills™ Assessment offered by the Educational Testing Service (ETS) is also 
under consideration, especially since off-campus testing is now offered (ETS, 2014). If used as 
students enter the program, it could provide additional diagnostic data concerning the 
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information literacy and technology skills of new students, thereby giving direction to course 
review and curriculum development. An additional avenue for research after the post-test 
analysis of data is identifying the data for students currently working in a library and comparing 
it to the overall pre/post-test analysis. This information could help determine if students are 
accurate in their perception of abilities, are over confident in their self-assessment, or are 
incapable of recognizing or are unwilling to admit their true abilities. 

Nonetheless, the MSLS survey is significant because it can direct student self-awareness 
and advisement according to expected learning outcomes and it can affirm or suggest curriculum 
content which, in turn, contributes to program improvement. Ultimately, the MSLS solidifies the 
overall assessment plan for accreditation. 
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